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COMMUTING INVOLUTIONS AND DEGENERATIONS OF ISOTROPY 

REPRESENTATIONS 

DMITRI I. PANYUSHEV 

Abstract. Let o-i and <72 be commuting involutions of a semisimple algebraic group G. 
This yields a Z2 x Z2-grading of g = \J\e{G), g ~ 0^ ^ gij, and we study invariant- 
' theoretic aspects of this decomposition. Let g((Ti) be the Z2 -contraction of g determined 

by CTi. Then both (T2 and CT3 := (Ti(72 remain involutions of the non-reductive Lie algebra 
0^' fl(o'i)- The isotropy representations related to (g(cri), 0-2) and (0(0-1), 0-3) are degenerations 

, of the isotropy representations related to (g, (J2) and (g, ca), respectively. We show that 

00 ! these degenerated isotropy representations retain many good properties. For instance, they 

always have a generic stabiliser and their algebras of invariants are often polynomial. We 
also develop some theory on Cartan subspaces for various Z2-gradings associated with the 
O: Z2 X Z2 -grading of g. 
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Introduction 

The ground field F is algebraically closed and char F = 0. Let ai and 0-2 be commuting 
involutions of a connected semisimple algebraic group G. This yields a Z2 x Z2-grading 
of g = Lie(G): 

(0-1) = ® Sy-' where g^^ = {x G g | ai{x) = & o-2(x) = (-l)^x}. 
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Since ai, (T2, and 0-3 = (Ji(J2 are pairwise commuting involutions, this decomposition pos- 
sesses an §3-symmetry, and following [24] we say that (0-1) is a quaternionic decomposi- 
tion. For, if (a, /3, 7) is any permutation of the indices (01, 10, 11), then [goo? da] C Qa and 
[da, S^] C Qj. Various problems related to this structure (in the setting of Lie groups or Lie 
algebras) has been studied before. Let G"'' be the fixed-point subgroup of a^. In [13], Mat- 
suki obtained fundamental results on (G"^\ G"^^) double cosets in G. The quotient variety 
G// (G"^^ X G"^2) has been studied by Helminck and Schwarz [7, 8]. In [24], Vergne applies 
quaternionic decompositions to the study of the Kostant-Sekiguchi correspondence. The 
conjugacy classes of pairs of commuting involutions are classified, see [9] and references 
therein. In [25], commuting conjugate involutions cri, (T2, as (triads) are used in the proof 
of Rosenfeld's conjecture on the existence of elliptic planes over the tensor product of two 
composition algebras. 

In this article, we consider some other aspects of quaternionic decomposition (0-1). This 
decomposition embraces several Z2-gradtngs of g and the subalgebras g'^' = Lie(G'^^). The 
involutions ui, (T2, are said to be big and the induced involutions of 0^^% g"^^ are said 
to be little. The same terminology applies to the corresponding Z2-gradings, symmet- 
ric spaces, and Cartan subspaces (= CSS for short). Our ultimate goal is to elaborate on 
invariant-theoretic properties of degenerations of isotropy representations involved. To 
this end, we develop some theory on the corresponding CSS and on the triples of com- 
muting involutions containing conjugate involutions. 

A dyad is a pair of different commuting conjugate involutions. Any dyad gives rise to 
a quaternionic decomposition with additional "symmetry". For a fixed cti, we provide a 
universal construction of dyads {ai, 02}, using automorphisms of order 4 (Prop. 2.5). This 
implies that every involution can be a member of a dyad. We also describe all possible 
conjugacy classes of involutions 0-3 = 0102 via the restricted root system of G/G'^^. 

or cTs. Let Cq be a CSS of ga (i.e., Cq is a little CSS). Then is contained in a CSS of 
0a © 07 or ga © g^ (i.e., of the (— l)-eigenspaces of two big involutions). If at least one 
such embedding appears to be an equality, then we call it a coincidence of Cartan subspaces. 
Clearly, there are totally six possibilities for such coincidences. This can also be expressed 
as the equality of ranks for certain little and big symmetric spaces. We prove two sufficient 
conditions for a coincidence of CSS: 

• If cTi is an involution of maximal rank, then Cio is also a CSS in goi © gio and Cn is also a 
CSS in goi © gn (Theorem 3.3). 

• If {cTi, (12} is a dyad, then cn is a CSS in both goi © gn and gio © gn (Theorem 3.7). 

Let Go* denote the connected subgroup of G with Lie algebra go* := goo © 0oi/ and 
01* := 010 © 011- Then (Go*:0i*) is the isotropy representation associated with the (big) 
symmetric space G/Gq*. We observe that each big symmetric space of G admits two de- 
generations to symmetric spaces of non-reductive algebraic groups; accordingly, each big 



COMMUTING INVOLUTIONS AND DEGENERATIONS OF ISOTROPY REPRESENTATIONS 3 

isotropy representation also admits two degenerations. The corresponding non-reductive 
Lie algebras are Z2-contractions of g in the sense of [16, 17]. Let Q{ai) be the Z2-contraction 
of determined by ai. For instance, 5(0-2) = g^o x fl^i/ where g^o = 0oo © 0io and 
Q*i = Boi © 011 are the eigenspaces of a2, and the superscript 'a' means that g^i is re- 
garded an abelian ideal. The corresponding connected group is G{a2) = G^o x exp(g"^). 
Remarkably, both ai and remain involutions of the group G{a2) and algebra 5(0-2). Then 
g(o"2)'^^ = goo X 001 =: ^/ which is the Z2-contraction of g"^^ = go* determined by a2, i.e., 
0(o"2)'^^ = 0*^^ (0-2). The (— l)-eigenspace of oi in 0(0-2) is the t-module denoted by gio oc gn. 
Likewise, starting with the Z2-contraction g{o'^), we still obtain gio^)"^^ = t, whereas the 
(— l)-eigenspace of cri in 3(03) is the ^-module denoted by gn oc gio. 

Let Goo be the connected subgroup of G with Lie algebra goo and Nqi := exp(gg^). Then 
K = Goo X ^01 is the identity component of both G{a2)'^^ and G{a-i)"^, and Lie(K) = 6. The 
symmetric spaces G{a2)/K and G{a3)/K can be regarded as degenerations of G/Gq^,, and 
we prove that G{a2)/K and G{as)/K are affine varieties. The A'-modules gio ocgn and 
gii oc gio have the same underlying vector space gi^, and afford the same action of the re- 
ductive subgroup Goo C K. But the actions of the unipotent radical Nqi c K are different 
(see Section 4 for precise formulae). We show that V := gio oc gn and V* := gn oc gio are 
dual i^'-modules. Thus, the isotropy representations (K : V) and {K : V*) are different 
degenerations of (Go* : gi*). 

Altogether, there are six degenerations of three big isotropy representations, and we 
study invariant-theoretic properties of these degenerated isotropy representations. This gen- 
eralises the setting of [17], where the coadjoint representation of Z2-contractions is stud- 
ied. The main idea behind our considerations is that, although K is non-reductive, {K:V) 
retains many good invariant-theoretic properties of (Go* : gi*). Coincidences of CSS fit in 
this setting as follows. Suppose that Cio is also a CSS in gio © gn. Then F[V^]^°i ^ F[gio], 
F[V]^ ~ F[gio]^°", and ¥[V] is a free F[V"] ^-module (Theorem 4.5). Moreover, if ai, (T2, and 
0-3 are conjugate, then all conceivable coincidences of CSS do occur, and such a simple 
description of invariants applies to all six degenerated isotropy representations. 

Most of the quaternionic decompositions have at least one coincidence of CSS. But this 
is not always the case, and examples are given in Section 3. In general, we prove that (i) 
the i^-module V always has a generic stabiliser, (ii) trdegF(l^)^ = trdegF(gu)'^°*, and 
(iii) ¥{V)^ is the fraction field of F[l^]^ (Theorem 4.10). Hence this degeneration does 
not affect the transcendence degree of fields and algebras of invariants. Furthermore, the 
algebra F[l^]^ is bi-graded and there is a 'contraction method' for obtaining /^-invariants 
on V and V* from Ffgi*]*^"*. Using this method, we describe F[V"]^ under less restrictive 
assumptions than a coincidence of CSS (Theorem 5.4). 

As a by-product of our methods, we prove polynomiality of the algebras of invariant 
differential operators for many degenerations of big symmetric spaces. We also show that 
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results of [5] on 'Invariant differential operators on Grassmann manifolds' can be better 
understood in the framework of quatemionic decomposition, see Section 6. 

Throughout, G is a connected semisimple algebraic group and g = Lie(G). 

- ng(a) (resp. 3g(a)) is the normaliser (resp. centraliser) of a subspace a C g. 

- the centraliser in g of x G g is denoted by 3g(x) or g^. 

- If X is an irreducible variety, then ¥[X] is the algebra of regular functions and F(X) 
is the field of rational functions on X. If X is acted upon by an algebraic group A, then 
F[X]-^ and ¥(X)-^ denote the respective ^-invariant functions. 

- If ¥[X]^ is finitely generated, then X//A := Spec (F[X]^). 

1. Generalities on involutions and isotropy representations 

Our main object is a connected semisimple algebraic group G with Lie algebra g. The set 
of all involutions of g is denoted by Inv(g). The group of inner automorphisms Int(G') ^ 
G/Z{G) acts on Inv(g) by conjugation. Two involutions are said to be conjugate, if they lie 
in the same lnt(G')-orbit. If a G Inv(g), then g = go © gi is the corresponding Z2-grading 
of g. That is, Qi = {x G g | (t{x) = (— We also say that (g,go) is a symmetric pair. 
Whenever we wish to stress that go and gi are determined by a, we write g*^ and g^^^ for 
them. We assume that a is induced by an involution of G, which is denoted by the same 
letter. The connected subgroup of G with Lie algebra go is denoted by Go, while the fixed- 
point subgroup of a is denoted by G"^. Hence Go is the identity component of G'^, and 
Go = G"^ if G is simply-connected. The representation of Go in gi, denoted (Go : gi), is the 
isotropy representation of the symmetric space G/Gq. 

We freely use invariant-theoretic results on the Go-action on gi or G/Go, as exposed 
in [11], [18]. A Cartan subspace (=CSS) is a maximal subspace of gi consisting of pairwise 
commuting semisimple elements. All CSS of gi are Go-conjugate and their common di- 
mension is called the rank of the symmetric space G/Gq, denoted rk (G/Go). 

The Cartan subspaces are characterised by the following property: 

(1-1) Suppose that a subspace a c gi consists of pairwise commuting semisimple elements. 
Then a is a CSS if and only 2/3g(a) fi gi = a [11, Ch. I]. 

Let c be a CSS of gi. Then every semisimple element of gi is Go-conjugate to an element of 
c and Gq-c is dense in gi. The generalised Weyl group for c, Wq, is defined to be No{c)/Zo{c), 
where No{c) = {s G Go | s-c C c} and Zq{c) = {s G Go | s-x = x Vx G c}. An element x G gi 
is called Go-regular if the orbit Go-x is of maximal dimension. Below, we summarise basic 
invariant-theoretic properties of the isotropy representation: 

- Let X G gi. The orbit Go-x is closed if and only if Go-x fi c 7^ 0; 

- X G gi is Go-regular and semisimple isi^) n gi is a CSS; 
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- Each fibre of the quotient morphism tt : gi — Qi//Gq = Spec (F[gi]"^") consists of 
finitely many Go"Orbits. The dimension of each fibre equals dim Qi — dim c. 

- The restriction of polynomial functions F[0i] — t- F[c] induces an isomorphism 
IF[si]'^" ^ F[c]^o (Chevalley's restriction theorem); 

- Wq is a finite reflection group in GL{t). Hence F[0i]'^" is a polynomial algebra and 
01 /Go is an affine space of dimension dim c; 

- F[gi] is a free Flgi]*^" -module. 

A torus 5 of G is a -anisotropic, if a{s) = s^^ for all s e S. A CSS is the Lie algebra of a 
maximal a-anisotropic torus. 

• We say that a E Inv(g) is of maximal rank if gi contains a Cartan subalgebra of g. 

As is well known, dim gi — dim go ^ rk g for any a, and the equality holds if and only if a 
is of maximal rank. 

• Let cr be an inner involution of g. We say that a is quasi-maximal if gi contains a 
regular semisimple element of g. 

Lemma 1.1. Given a G Inv(g), the subspace gi contains a regular semisimple element ofg if and 
only if it contains a regular nilpotent element ofg. 

Proof For e G gi nilpotent, there is a normal s[2-triple {e, h, /}, i.e., such that e, / G gi and 
/i G go [11, Prop. 4]. If e is regular, then e + / is regular semisimple (conjugate to h). 

Conversely, suppose that x G gi is regular semisimple. Then Gq-x is a Go-orbit of 
maximal dimension and there is also a nilpotent element y G gi whose Go-orbit has the 
same dimension. Since dimG-2; = 2dimGo--2 for all z E Qi [11, Prop. 5], we see that y is 
regular in g. □ 

Remark 1.2. It follows from Lemma 1.1 and [15, Theorem 2.3] that the quasi-maximal in- 
volutions, as well as involutions of maximal rank, form a single lnt(G)-orbit. 

Let ko (resp. ki) denote the number of even (resp. odd) exponents of g, so that ko + ki = 
rkg. 

Proposition 1.3. (i) If a is of maximal rank, then rkgo = ki. 

(ii) If (J is quasi-maximal, then dim go — dimgi = ko — ki and dim(gi/Go) = ki. 

Proof Part (i) is proved in [14, Lemma in p. 1473]. 

In view of Lemma 1.1, part (ii) follows from [15, Theorem 3.3] with m = 2. □ 

The quasi-maximal involutions and involutions of maximal rank coincide if and only if 
the Weyl group of g contains —1 if and only if all exponents of g are odd. The property of 
having maximal rank is inheritable in the following sense. 

Lemma 1.4. Let x E gibe semisimple. If a is of maximal rank, then the restriction of a to 3g(a;) 
and [3g(a;),3g(a;)] is also of maximal rank. 



6 



D.PANYUSHEV 



Warning, The corresponding assertion for quasi-maximal involutions is not always true. 

2. QUATERNIONIC DECOMPOSITIONS, TRIADS, AND DYADS 

Let 0"! and 02 be different commuting involutions of g. The corresponding Z2 x Z2-grading 
of is: 

(2-1) = ® 0ii> where g^^ = {x G g | 0-1 (x) = & a2(x) = (-l)^x}. 

ij=0,l 

We also say that it is a quaternionic decomposition of g (determined by (Xi and (T2). Let 
0-3 := (Ti(j2. The involutions ui, (72/ and (T3 are said to be big. The involutions induced on 
the fixed-point subalgebras of big involutions are said to be little. The same terminology 
applies to the corresponding Z2-gradings, CSS, etc. Thus, associated with (2-1), one has 
three big and three little isotropy representations. It is convenient for us to organise the 
summands of (2-1) in a 2 x 2 "matrix": 

Soo i Soi 

(2-2) S = -giot All ■ 

(72 

Here the horizontal (resp. vertical) dotted line separates the eigenspaces of cti (resp. (T2), 
whereas two diagonals of this matrix represent the eigenspaces of 0-3. Hence the first row, 
first column, and the main diagonal represent the three little Z2-gradings (of g°^S g"^^, and 
g*^^, respectively). 

We repeatedly use the following notation for the eigenspaces of ai and (T2: 
go* := 000 © 001/ Qu '■= 010 © 011/ 0*0 := 0oo © 0iO/ 0*i := 0oi © 0ii- 

Likewise, the connected subgroup of G corresponding to go* is denoted by Go*/ etc. 
Following Vinberg [25, 0.3], we say that a triple {cri, 0-2, 0-3} C Inv(g) is a triad if (Ti(T2 = 0-3 

and all these involutions are conjugate. A complete classification of triads is obtained in 

[25, Sect. 3]. Obviously, triads lead to the most "symmetric" quaternionic decompositions. 

Below, we are interested in more general (hence, less symmetric) decompositions. 

Definition 1. We say that {cti, 0-2} C Inv(g) is a dyad if cri, 0-2 are conjugate and aia2 = o"20"i. 

That is, we do not require that the third involution 0-3 = aia2 is necessarily conjugate to ai. 
Note that the product of two conjugate involutions (not necessarily commuting) is always 
an inner automorphism of g. For, if a2 = Int(5')-cri-Int(5'~^), then cricr2 = lnt{ai(g)g~^). 
Therefore, any triad consists of inner involutions. A member of a dyad can be an outer 
involution. But the third involution, 0-3, is necessarily inner. 

We begin with describing the dyads containing involutions of maximal rank. 

Proposition 2.1. Let fibea semisimple inner automorphism of g. Then there exist involutions of 
maximal rank 1!} and such that jj, = 
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Proof. By assumption, /i = Int(s) for a semisimple element s G G. Choose an involution 
of maximal rank such that s belongs to a ??-anisotropic maximal torus T. Clearly, the 
mapping T ^ T , t ^ 'd{t)t~^ = is onto. Therefore, s = 'd{g)g^^ for some g E T. Set 
i!)' = liit{g) ■'d-lnt{g~^) . It is another involution of maximal rank, and a direct computation 
shows that = lnt{'d{g)g-'^) = /i. □ 

Proposition 2.2. 

1) Suppose that jj, g Inv(g) zs inner. Then there are commuting involutions of maximal rank, -d 
and t?', such that /i = {}■{}'. Moreover, i!) and induce an involution of maximal rank ofg^. 

2) For commuting involutions of maximal rank, i!} and ■(}', the following conditions are equiva- 
lent: 

(i) the inner involution jj, = is quasi-maximal; 

(ii) 'd'\Q^ is an involution of maximal rank of q^; 

(iii) is an involution of maximal rank ofg"^'. 

Proof. 1) Suppose that fi = Int(s). Then G Z{G) and Int(s) = Int(s~^). Choose and 
as in the proof of Proposition 2.1. Then = Int(s) and = Int(s~^). Thus, t? and -d' 
commute. Consider the quaternionic decomposition of g determined by (i), ■&'): 

000 i 001 

Q = (B and the corresponding 'dimension matrix' 

010 : 011 1 O 

(i.e., X = dim 000/ etc.). Let Uhe a maximal unipotent subgroup of G. Since i!) and 13' are of 
maximal rank, 

x + y = x + u = dim U, y + v = u + v = dim U + Tkg. 

Therefore, u = y and v — x = ikg. Consequently, dimgn — dim goo = rkg = rkg^. Hence 
0^^ = 000 © 011 is a Z2-grading of maximal rank. 

2) (i)^(ii),(iii). By the assumption and Proposition 1.3(ii), we have x + v — 2y = ko — ki. 
Consequently, x = (dimf/ — ki)/2 and y = (dimf/ + ki)/2. Hence, y — x = ki = ikg^ = 
rk g'^'. Therefore, the induced Z2-grading of g'^ (or q^') is of maximal rank. 

Conditions (ii) and (iii) are equivalent, since the dimension matrix is symmetric. 

(ii)^(i). By assumption, goi contains a Cartan subalgebra of g''. Furthermore, the fixed- 
point subalgebra of any involution (e.g. q^) contains a regular semisimple element [18, 
Lemma 5.3]. Hence goi contains a regular semisimple element of g. Thus, goi © gio = 0i'^^ 
contains a regular semisimple element of g, i.e., fi is quasi-maximal. □ 

Definition 2. A triple {-i?, -i?', /i} C Inv(g) is said to be canonical, if d and -d' are involu- 
tions of maximal rank and ^"d' = /i is quasi-maximal. The corresponding quaternionic 
decomposition is also called canonical. 



X 


y 


u 


V 
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Remark 2.3. By Proposition 2.2, all little involutions involved in the canonical decompo- 
sition are of maximal rank. Furthermore, gn contains a Cartan subalgebra of g. Another 
interesting feature is that each of three subspaces gi^, g^i, and gio © goi meets all nilpotent 
G-orbits in g. This follows from results of Antonyan [1]. In Section 5, we obtain further 
results on canonical decompositions. 

As we shall shortly see, every involution is a member of a dyad. This raises the natural 
question: 

What (conjugacy classes of) involutions can occur as products of two members of dyads con- 
taining given cr G Inv(g)? 

For instance. Proposition 2.2(1) asserts that every inner involution as product from a 
dyad of involutions of maximal rank. We give below a general answer in terms of the 
reduced root system for a. The following is a recipe for constructing dyads. 

Proposition 2.4. For any ai G Inv(g), there is (p G Int(G') such that (f)'^ = id, 0-2 := 0cri</)~^ 
commutes with o\, and o\a2 = 0^. In particular, {ai, 0-2} is a dyad. 

Proof. Let 5* be a one-dimensional cri-anisotropic torus in G and s G S be an element such 
that G Z{G). Letting (p = Int(s), one has 0^ = id, cri^cri = (p~^, and the remaining 
relations are easily verified. □ 

Remarkably, all the dyads are obtained in this way! 
Proposition 2.5. Suppose that {ai, 0-2} is a dyad. Then there exists (j) G Int(G') such that cj)'^ = id, 

(T2 '■= 0cri0~^/ l^nd cricr2 = 0^- 

Proof. Suppose that (T2 = Int(5()(TiInt((7~^) for some g E G and hence 01O2 = \nt{ai{g)g~^). 
Set g = ai{g)g~^. Since ai and 0-2 commute, we have g'^ G Z{G), o-i{g) = g~^, and 0-2 (^) = 
g^^. By [18, Prop. 6.3], the property that g is semisimple and g = ai{g)g^^ guarantees us 
that g is contained in a ai -anisotropic one-dimensional torus. It follows that there exists 
s G G such that = g and still o'i{s) = s~^. Then = Int(s) will do. An easy verification 
is left to the reader. □ 

Let C be a maximal ui -anisotropic torus (hence Lie(C) is a CSS in qu). Recall that a 
restricted root of G is any non-trivial weight in the decomposition of g into the sum of 
weight spaces of G. Write for the set of all restricted roots. That is, 

(2-3) = S''©(007)- 

We use the additive notation for the operation in X{G), the character group of G, and re- 
gard as a subset of the vector space X{G) ®z The set satisfies the usual axioms of 
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finite root systems [6]. The notable difference from the structure theory of split semisim- 
ple Lie algebras is that the root system can be non-reduced and that = dimg^ 
(7 G $'^) can be greater than 1. 

The (universal) construction of dyads described in the proof of Prop. 2.4 means that, 
for given ai, all possible involutions are obtained in the following way. Take a one- 
dimensional torus S C C, consider the corresponding Z-grading of q: 

= 0/(0, 

and then set Qq = ©jezS'^(2i) and 31 = ©igz0'^(2« + !)• Alternatively, this can be expressed 
as follows. Choose a linear form i on X(C) ®z M that takes integral values on (One 
should assume that ^ 2Z.) Define $f to be the set of all restricted roots 7 such that 

£(7) is odd. Then 0^g(j,c g-y is the (— l)-eigenspace of 0-3. 

Remark 2.6. If is reduced, then all possible involution 0-3 are associated with the inner 
involutions of the semisimple Lie algebra with root system 

Example 2.7. If ai is of maximal rank, then C is a maximal torus of G and is the usual 
root system of g. Here all one-dimensional tori are at our disposal, hence as can be any 
inner involution (which is already known from Prop. 2.2). 

Example 2.8. Let ai be an involution E IX, i.e., g is Eg and g"^^ is E7 x Ai. Here dim C = 4 

I 8, if 7 short 

and $^ is of type F4; for 76$, one has = < , see [6, Ch. X, Table 6] or 

1^1, if 7 long 

[26, Table 9]. The Lie algebra F4 has two (conjugacy classes of) inner involutions, and this 
leads to two possibilities for a^. Using information on the multiplicities of restricted roots, 
one easily computes dim g'^^, which allows us to identify 173. The answer is that involution 
F I (resp. FN) leads to the involution of g with dimg'^^ = 120 (resp. 136), i.e., as is either 
E VIII or E IX. 

3. Comparing Cartan subspaces 

As is explained above, the quatemionic decomposition (2-2) embraces six Z2-gradings. In 
this section, we compare CSS for little and big Z2-gradtngs. 

For (ij) 7^ (00), let Cij be a CSS of Qij; that is, a CSS related to the little Z2-gradtng Qoo®Qij. 
There are also CSS for three big involutions: 

c gu, c^i c g^i, c^,i_^ c g^,i_* := goi © gio- 

Each little CSS can be included in two big CSS. E.g., because gio C Qi^, and gio C g^,i— one 
can choose Cartan subspaces Cu and c^,i_^ such that Cio C Cu and Cio C c^,i_^. If at least 
one equality occurs among all such inclusions, then this will be referred to as a coincidence 
of CSS (for a given quatemionic decomposition). We obtain two sufficient conditions 
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for such a coincidence to happen and provide examples of quaternionic decompositions 
without coincidences of CSS. 

We begin with a preparatory result. Let k denote the Killing form on g. Set mij = 
[dijiQij] C Soo for (ij) 7^ (00). Clearly, rtiij is an (algebraic) ideal of the reductive algebraic 
Lie algebra goo- Therefore rtiij is also reductive and kI^^-^ is non-degenerate. 

Lemma 3.1. Suppose that Qu = 0. Then 

(i) [moi,0io] = [tnicSoi] = 0; 

(ii) K(moi, rriio) = and moi fi rriio = {0}; 

(iii) rriio © 0io ^nd rrioi © Soi disjoint ideals ofg. 

Proof. Everything follows from the Jacobi identity, the relation [gio, 0oi] = 0, and the fact 
that is non-degenerate. □ 

Corollary 3.2. If go* = 0'^^ does not contain proper ideals of g and gu = 0, then goi = as well. 
(In particular, this applies if ai is of maximal rank.) 

Proof. Since ai is non-trivial, we have gio 7^ 0. As moi © 0oi is a proper ideal of g lying in 
00*/ it must be zero, i.e., 001 = 0. □ 

Now, we are in a position to prove our first result on the coincidence of CSS. 

Theorem 3.3. Let ghe a semisimple Lie algebra and {cri, 0-2, o"3} 1 triple of involutions of g such 
that cri(j2 = (Ta. Suppose that ai is of maximal rank. Then (1) any CSS Cn C 0ii is also a CSS in 
0^1, i.e., for (T2; (2) any CSS Cio C 0io is also a CSS in 010 © 0oi, i.e., for 0-3. 

Proof. Obviously, (2) is obtained from (1) if we permute (J2 and as. Therefore, it suffices to 
prove the first assertion. 

Set I = 3g(cii) and s := [I, I]. Then [ is a (ai, (T2)-stable Levi subalgebra of 0. Let 3 be the 
centre of [, so that t = s © 3. By construction, Cn C 311. Furthermore, since Cn is a CSS of 
011, we have 3g(cii) n 011 = Cn, i.e., Su = and Cn = 311. 

As (Ti is of maximal rank, there exists a Cartan subalgebra t C such that Cn C t C 0i^. 
Then [3, t] = and therefore 3 C t C 0i*. By Lemma 1.4, the restriction of ai to the 
semisimple algebra s is still of maximal rank. Applying Corollary 3.2 to s in place of 0, 
we conclude that Sqi = 0. Because 3 C gu, this also means that loi = 0. Thus, 

3g(cii) n 0^1 = 3g(cii) n 011 = Cn, 

which is exactly what we need, in view of (IT). □ 

Alternatively, coincidences of CSS in Theorem 3.3 can be expressed in terms of the rank 
of symmetric spaces. 

Corollary 3.4. Ifi!} and a are commuting involutions and -d is of maximal rank, then 
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rk (G/G") = rk {G'^^/G^ n G") and rk {G/G^'') = rk (G'^/G^ n G''") . 
The following readily follows from the symmetry between ai and a2'. 

Corollary 3.5. Suppose that both ai and a2 are of maximal rank. Then 

(i) any CSS Cn C gn is also a CSS in or in g^i, i.e., for ai or 0-2; 

(ii) any CSS Coi C goi is also a CSS in qiq © goi, i.e., for a^, 

(iii) any CSS Ciq d Qiq is also a CSS in qiq® QQi,i.e.,for a^. 

Remark 3.6. If both oi and 02 are of maximal rank, then Cn is a Cartan subalgebra of g (use 
Prop. 2.2(1)). This provides another explanation for part (i) in Corollary 3.5. 

Our second result on the coincidence of CSS concerns arbitrary dyads. 

Theorem 3.7. Let gbe a semisimple Lie algebra and {ai, (T2, 0-3} a triple of involutions of q such 
that (Ticr2 = cTs- Suppose that {ai, 02} is a dyad. Then any CSS Cn C gn zs also a CSS in gi^ or 
g^i, i.e., for ai or 0-2; 

Proof. As in the proof of Theorem 3.3, we consider the (ai, o'2)-stable Levi subalgebra 

30(cii) =: l = s©3, 

where s is semisimple and 3 is the centre of [. Since Cn is a CSS in gn, we have [n = Cii(= 
3ii), so that Sii = {0}, and our task is to prove that [qi = tio = {0}. 

Since ui and (T2 are conjugate, 0-2 = Int((7)(7ilnt(5'"^) for some g E G. Then 0-3 = Int(^), 
where g = ai{g)g~^. Since aia2 = (J2<^i and Int(^) = Int(^^^), we also have g = g~^a2{g). 
The choice of g is not unique, and we are going to demonstrate that g can be chosen to 
have some extra properties. 

Let X G Cii be generic. In particular, x G g'^'^, i.e., lnt[g)-x — x. Making use of two 
expressions for g, one easily computes that 

ai{lnt{g^^)-x) = —lnt{g~^)-x and a2{lnt{g)-x) = —lnt{g)-x. 
In particular, 

(3-1) Intig-^yx C gu = gS"'^ and lnt{gyx C g.i = gS"'^ . 

Recently, M. Bulois [3, Prop. 6.6] noticed that any Z2-grading g = go ©gi has the following 
property: 

Ifxegi is semisimple, then G-x fi gi = Gq-x. 
Applying this, say, to the first inclusion in (3-1) shows that the identity component of G"^ 
contains p such that \nt{g~^)-x = Int(p)-x, i.e., gp G Zg{x), the centraliser of x in G. 

Replacing g with gp does not affect g and (72- Therefore, we may assume that our ini- 
tial g lies in Zg{x). Note that this group is always connected. So far, we did not use 
the assumption that x is generic in cn. For generic x, we have 3g(x) = 3g(cii). Hence 
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L\e{ZG{x)) = I, and we will write L in place of Zg{x). Since g G L, the involutions ai\s and 
(72 Is are conjugate with respect to Int(S'), where S := (L, L) is connected and semisimple. 
Note that Lie(5) = s. 

Let us prove that Soi = Sio = {0}. Since Sn = {0}, it follows from Lemma 3.1(iii) that 
qio = Sio©[5io,Sio] and qoi = Soi©[soi,Soi] are disjoint ideals of s. Thus, 3*^1 D qoi,s'^^ D qio/ 
and 3°'^ and s"^^ are conjugate with respect to S. It is only possible if qio = qoi = {0}, i.e., 
5io = 5oi = {0}. This completes the first part of our programme. 

The second part deals with the centre of [. Let the a Cartan subalgebra of 0°^^ = Soo ©Sii 
that contains Cn. Since 0-3 is inner, t is actually a Cartan subalgebra of q. As t C 3g(a;), t 
must contain 3. Therefore, 3 C 0°^^ and 301 = 3io = {0}. 

We have proved that toi = ^lo = {0}. Hence 3g(cii) n = 3g(cii) fi 0^1 = Cn, and we are 
done. □ 

Corollary 3.8. If {ai, (T2, 0-3} is a triad, then every little CSS is also a big CSS in two possible 
ways. (That is, six coincidences of CSS occur.) 

Again, the coincidence of CSS can be expressed in terms of ranks of symmetric spaces: 
Corollary 3.9. If {ai, ^2} is a dyad, then rk (G'^^'^^ jQ^i r\ rk [G/G"^). 

Theorems 3.3 and 3.7 show that, for many triples of commuting involutions, there is 
a coincidence of CSS. However, this is not always the case, and we provide below two 
examples of commuting triples without coincidences of CSS. 

Example 3.10. Let g be a semisimple Lie algebra and a an involution of g with the corre- 
sponding Z2-grading = 0o © 0i. We specify the requirements on a below. Set g = © 
and define three involutions of as follows: 

(Ti(a;i,X2) = (X2,xi), Cr2(xi,X2) = ((T(xi),Cr(x2)), (T3 = (Ji(T2. 

Then q"^ = A{g), the diagonal; 0'"^ = 00 © 0o; 0"^' = {(x, a{x)) | x e g}. 

Set A_(M) := {(m, — m) | m G M} for any subspace M c g. Then the corresponding 

quatemionic decomposition is: 

A(0o) \ A(gi) 
A_(0o) i A_(0i) 

0"2 

Let c be a CSS of 0i and dim c = r. Let t be a Cartan subalgebra of and to a Cartan 
subalgebra of go. Both little and big CSS can explicitly be described. We have 

coi = A(c) C A(gi), Cio = A_(to) C A_(go), Cn = A_(c) C A_(gi) 

and 

cu = A_(t) C A_(g), c^i = c © c C gi © gi, c^,i_,, = {t, -a{t) \ t et} C {x, -o-{x) \ x e q}. 
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The absence of coincidences means that each big CSS has strictly bigger dimension than 
either of two little CSS that can belong to it. This yields the following conditions: 

r = dim Cn < dim Ci^, = rk g, rk Qq = dim Cio < dim Cu = rk g, 
r = dim Cn < dim c^i = 2r, r = dim Cqi < dim c^i = 2r, 
rk go = dim Cio < dim c*,i_* = ikg, r = dim Cqi < dim Cv,,i_^ = rk q. 

All this amounts to the inequalities rk go < rk g and r < rk g. Both are satisfied if and only 
if cr is outer and not of maximal rank. E.g., such an involution a exists if g is a simple Lie 
algebra of type A2„+i or Da^+i or Eg. 

A drawback of Example 3.10 is that g is not simple. Actually, it is not easy to discover 
such an example for simple Lie algebras. A coincidence of some little and big CSS is rather 
a rule, than exception. For instance, any triple of commuting involutions of exceptional 
Lie algebras contains either a dyad or an involution of maximal rank, see [9, Table 1]. 
We can also prove that there is always a coincidence of CSS for the triples of commuting 
involutions of g = s[„. The following example concerns algebras of type D^r. 

Example 3.11. g = SO2N and G = S02n- Consider three involutions of g with the following 
fixed-point subalgebras: 

For even, Int(G) contains two conjugacy classes of involutions with the fixed-point 
subalgebra glj^, and we will arrange that cri,a2 belong to different conjugacy classes. 
Namely, assume that g is represented by the matrices of order 2 that are skew-symmetric 
w.r.t. the antidiagonal and define the a/s using certain diagonal matrices, as follows: 

0-1 = Int (diag(z, . . . , z, -i, . . . , -i)); 

(72 = Int(diag(i, 

m 

0-3 = (Tia2 = lnt(diag 

Here i = y/—T. One easily presents this quaternionic decomposition in the matrix form. 
Taking the eigenspaces of cti and (T2 partition the matrices in g in 16 blocks, and we indi- 
cate the subspace g^j which each block belongs to: 

/ 000 001 010 011 \ 
001 000 011 010 
010 011 000 001 
\011 010 001 000 / 

The diagonal blocks consists of square matrices of order m,n,n,m, respectively. Hence 
the framed block consists of rectangular matrices of shape nxm. Then one computes that 
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dimcoi = dimcio = min{n,m}, dimcn = [n/2] + [m/2]; and 
dim Cu = dim c^i = [n + m/2], dim c^,i_^ = 2 min{n, m}. 

If n, m are odd, then cri, (T2 are not conjugate. Furthermore, if n, m are odd and n ^ m, 
then there is no coincidence of CSS, since [n/2] + [m/2] < [n + m/2] and min{n, m} < 
[n + m/2]. 

Remark 3.12. A coincidence of Cartan subspaces has the following invariant- theoretic 
meaning. Suppose that Cio C gio is also a CSS in gio © gn = gi*. Then, in view of 
Chevalley's restriction theorem, the natural restriction homomorphism 

Q ■ F[su]««* -> F[gio]''"'' 

is injective, and it makes F[gio]'^"° a finite F[gi^]'^"* -module. The degree of the finite ring 
extension equals the ratio of orders of the corresponding generalised Weyl groups. It 
should be noted that F[gio]'^'"' is always a finite ^(F[gi^]'^"*)-module. That is, the point is 
that such a coincidence of CSS implies the injectivity of q. 

Putting this in the geometric form, we obtain the commutative diagram 

i 

Sio 01* 

; _ ; 

where the vertical arrows are quotient morphisms, % is the embedding, and the morphism 
i is finite and surjective. 

4. Degenerations of symmetric spaces and isotropy representations 

For any a G Inv(g) with Z2-grading g = go © gi, there is a non-reductive Lie algebra, 
which is a contraction of g. Namely, the semi-direct product g((T) = go x is called a 
lj2-contY action of g (with respect to cr). Here the superscript 'a' means that the go-module 
gi is regarded as an abelian ideal of g(cr). Then g° is also the nilpotent radical of g((T). 
The corresponding connected group G{p) is a semi-direct product of Go and the abelian 
unipotent radical exp(gf ), i.e., G{o) = GqV. exp(g"). 

Invariant-theoretic properties of the adjoint and coadjoint representations of G{o) have 
been studied in [16, 17]. By [16, Thm. 6.2], the algebra F[g((j)]'^^'^^ is always polynomial. In 
[17], it is proved that the algebra F[g((T)*]'~^^'^^ is polynomial in many cases. There is also a 
useful method of "contraction" of G-invariants. Namely, to any homogeneous / G F[g]'^ 
one can associate an element of either F[g((T)]'^^'^^ or F[g((T)*]'^^'^^ [17, Prop. 3.1]. In the 
context of quaternionic decompositions, we may extend the scope of this method beyond 
the (co)adjoint representations. 

Recall that we work with a quaternionic decomposition 
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000 



001 



= ® - 0"! 

010 i 011 

0-2 

and (73 = (Ticr2- Consider the Z2-contraction of 0°^^ with respect to a2 (or 0-3, which is the 
same), that is, set ^oi := 0'^^(o'2) = 0oo x 0oi- The vector space Qu can be regarded as 
toi-niodule in two different ways. In both cases, the subalgebra goo C ^oi acts on as it 
was in g. Let x G goi and (yo.yi) G 0io © Sii- For the action of the abelian nilpotent radical 
001 C toi/ the two possibilities are: 

a) x-{yo,yi) = (0, [x,yo]); 

b) x-{yo,yi) = {[x,yi],0). 

The ^oi-iTiodules obtained in this way are denoted by gio oc gn and gn oc gio, respectively. 
From a slightly different angle, these possibilities can be realised as follows: 
a') Take the Z2-contraction of g with respect to 0-2, i.e., g(cr2) = Q*o ^ 0"i- Identifying g and 
g(cr2) as vector spaces, we notice that the linear operators ai and 0-3 remain involutions 
of the Lie algebra 2(0-2) ■ Taking the eigenspaces of ai in g((T2) yields g(cr2)'^^ = ^oi and 
qW2)['"^ = 0ioocgii. 

b') Likewise, starting with g(a3), we end up with Qia^y^ = toi and 0(0-3)1'^^^ = 0ii oc gio. 

We have G{a2) = G^o x exp(g^^) and both G^o and g^,! are cri-stable. It follows that cri 
can be lifted to an involution of G{a2) and the identity component of G{a2Y^ is -K'oi = 
Goo X exp(gQ^) =: Goo x ^oi- (Likewise, i^^oi is the identity component of Gia^iY^.) The 
exponential map exp : goi — A^oi is an isomorphism of varieties and the action of A^'oi is 
given by 

(4-1) exp(x)-(yo,2/i) = (yo,yi) + x-(yo,yi) = (2/0,^/1 + [x^yo]) for (2/0,2/1) e gioocgn, 
exp(x)-(2/i, yo) = {yi, yo) + x-{yi,yo) = (yi, yo + [x, yi]) for {yi,yo) G 0ii oc gio- 

Note that Goo is a subgroup of both Koi and Go*, and the action of Goo does not vary under 
the passage from the Go* -module gi* to the i^oi-modules gio oc gn or gn oc gio. 
Summarising the previous discussion, we get the following: 

Claim 4.1. 1) The involution ai E Inv(g) can be regarded as involution of g{a2) and of G{a2)- 
The group A'oi is the identity component ofG{(j2y^ and the corresponding isotropy representation 
is (A'oi : gioocgii). 

2) Likewise, Kqi is the identity component ofG{a'sy^ and the corresponding isotropy represen- 
tation is {Kqi : gii ocgio). 

3) Both representations can be understood as different degenerations of (Go* : gi*). The sym- 
metric spaces G{(X2)/Koi and G{a^) /Kqi can be regarded as different degenerations ofG/G^,o- 



Lemma 4.2. The K^i-modules gio oc gn and gn oc gio are dual to each other. 



16 



D.PANYUSHEV 



Proof. Ify = {yo,yi) e 010 oc gn and 2; = {zi.zq) e gn oc 010, then (y, 2) t-^ K{yo, zo)+K{yi, zi) 
is the required A'oi -invariant pairing. □ 

Lemma 4.3. The homogeneous space G{a2) /Kqi is affine. 

Proof. Let denote the unipotent radical of an algebraic group A. By construction, 

G(a2)" = exp(g^i) and (i^oi)" = exp(ggi). Hence (A^oi)" C G'(a2)". By [2, Cor. 2], this 
condition is sufficient for G{a2) / Kqi to be affine. □ 

Hint. Using the notation gio oc gn or gn oc gio always means that the vector space gi^ is 
regarded as a -Aoi -module in the prescribed way. 

Similar notation is used for the other Z2-contractions of g and their involutions. For 
instance, g( 0-1) = Q(^(j^y2 =QQQtKQ% =: ho, q{<^i)["^^ = goi oc gn, and g(cr3)S'''^ =giiocgoi. 
Then A^'io = exp(g'^Q) and Kiq = Goo 1^ ^10 acts on goi ocgn or gn ocgoi- In this way, we 
obtain 6 new isotropy representations related to 3 possible Z2-contractions of g. 

Example 4.4. The adjoint and coadjoint representations of any Z2-contraction can be ob- 
tained as a special case of this construction. Given a E Inv(g), consider three involutions 
of g = g © g as in Example 3.10: 

ai{x,y) = {y,x), a2{x,y) = {a{x),a{y)), and (T3 = criO-2- 

Then g(a2)'^^ = g(a) and g(cr2)i'^^^ is isomorphic to the adjoint module of g{a); whereas 
g(a3)'^^ = g(a) and g(cr3)['^^^ is isomorphic to the coadjoint module of g(cr). 

Convention. Whenever it is notationally convenient, we will expose our results in the 
most symmetric form, i.e., for an arbitrary permutation of indices (10, 01,11). Otherwise, 
we stick to our sample choice with K = Kqi and the K -module = gio oc gn. 

Let {a, /3, 7) be a permutation of (10, 01, 11). The representation of Kfs = Goo x in 
Va-y := ga OC g^ is said to be a degenerated isotropy representation. There is a special situation 
in which the algebra F[V;^]^^'9 can explicitly be described. Since ¥[Va^]'^f = (¥[Va^]^f)'^°°, 
the first step is to describe the algebra F[Vq^]^''. It follows from the appropriate analogue 
of (4-1) that the dimension of any A'/j-orbit in Va-y is at most dimg^ and if p : Va-y -> Qa is 
the projection along g^, then F[g„] ^ p*(F[g«]) C ¥[Va^]^^. 

Theorem 4.5. Suppose that there is y E Qa such that [y, g^] = g^. Then 

(i) F[K.^]^'' ^ F[g,], 

(ii) F[Kj^]'^'3 ~ F[gQ,]'^'"' is a polynomial algebra, 

(iii) F[Vq^] is a free ¥[Vay]^'^ -module. 

Proof, (i) Since g^ C Va-y is A^/3-stable, the projection vr : Vaj — )■ Va-y/g-y ^ Qa is 
equivariant. Moreover, the induced A'^g-action on Va-y/g-y is trivial. In order to prove that 
TT is the quotient by we use the Igusa lemma, see e.g. [16, Lemma 6.1]. Since vr is 
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onto and Va^/g^ is normal, it suffices to prove that generic fibres of n are A^/j-orbits. If 
[y^ 0/3] = 07 ^rid we identify y with (y, 0) G V^q^, then the orbit N^-y = y+ [y, g^] is precisely 
a fibre of vr. The appropriate analogue of Eq. (4-1) shows that dim A'^a ■(?/', y") = dim[?/', g^]; 
in particular, it depends only on y' E g^- Consequently, there is a dense open subset 

C Qa such that [y, g/3] = for all y e Vl. Finally, identifying with an open subset in 
Va-f/g-f, we see that Tr~^{y) is a sole orbit for all y eVI. 

(ii), (iii) This follows from (i) and the corresponding properties of the algebra of invari- 
ants for the isotropy representation (Goo : 0a)- D 

The above proof shows that F[Vq,^]^'3 = p*(F[gQ,]) if and only if there is y e such that 
[y, 0/?] = 07 if and only if max^gv-c-y dim Np-v = dim g^. 

We will see in a moment that the hypothesis of Theorem 4.5 is equivalent to a coinci- 
dence of CSS. 

Lemma 4.6. For any quatemionic decomposition and x G go,, we have dim[g/3, x] = dim[g^, x]. 

Proof. Set dij = dim3g(x)jj. Applying [11, Prop. 5] to one little and one big Z2-grading, we 
obtain: 

da~doo = dimg„ - dim goo , 

+ d^) - {doo + = dim(g„ © g^) - dim(goo © g/3) • 

Taking the difference yields d^ — dfs = dimg^ — dimg^, as required. □ 

Proposition 4.7. There exists x G g^ such that [gp, x] = g^ if and only if any CSS Cq, C go, is 
also a CSS in g^ © g^; 

Proof. . The set of elements x G ga having such property is open. If x is Goo-regular 
and semisimple, then 3g(x)Q, is a CSS in g^. By Lemma 4.6, the assumption implies that 
= {0}- Thus, the CSS ■= dsi^)» the property that 3g(ca) fi (g„ © g^) = 3g(x) fi 
(ga © g7) = Ca, i.e., Ca is a CSS in g^ © g^. 

"<^=". Reversing the preceding argument shows that any Goo-regular semsimple ele- 
ment X G Cq has the required property. □ 

This proposition yields a simple necessary condition for coincidence: 
Corollary 4.8. Ifca is also a CSS in ga © g-y, then dimg/3 ^ dimg^. 

Combining our previous results, we get an explicit description of the algebra of invariants 
for certain degenerated isotropy representations. 

Theorem 4.9. Let gbe a semisimple Lie algebra and {ai, (T2, 0-3} a triple of involutions of g such 

that CTi(T2 = CTS- 

(i) Suppose that ai is of maximal rank. Then 

F[giiocgoi]^°°"^^" ~F[gn]^«° and F[gio oc goi]^°°"^" ~ F[gio]«»°. 
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(ii) Suppose that {(Xi, 0-2} is a dyad. Then 

F[0iiocgio]^»°"^°i ^Fisn]^"" and F[0n oc goi]^""^'^"' - Flfln]^"". 

(iii) Suppose that {ai, (T2, cs} is a triad, then F[0q x 0^]*^"°^^'^ ~ ¥[Qa]'^'^° for all permutations 
(a, /3, 7) 0/(01, 10, 11). 

In all previous cases, x 07]^^ = ^[Qa] and ¥[Qa x 07] is a free F[0q, ix Q^]^°°^^f^ -module. 

Proof, (i) Combine Theorem 3.3, Theorem 4.5, and Proposition 4.7. 

(ii) Combine Theorem 3.7, Theorem 4.5, and Proposition 4.7. 

(iii) Combine Corollary 3.8, Theorem 4.5, and Proposition 4.7. □ 

If [0/3, x] is a proper subspace of 0^ for all x G 0a, i.e., the maximal dimension of Np- 
orbits in Va^ is less than dim0^, then a general description of F[Va7]^'^ (and hence of 
F[Vq^]^'3) is not available. Nevertheless, the K^-module Va^ always retains some good 
properties of initial isotropy representation of a reductive group. 

Below, we consider the problem of existence of a generic stabiliser for the linear action 
of an algebraic group K on V . Recall that a subalgebra q C Ms a generic stabiliser if there 
exists a dense open subset Vl (Z V such that the stabiliser ^" = [k E t \ k-v = 0} is K- 
conjugate to q for all v G f2. We then write q = g.s.(A':V") and the elements of f2 are called 
K-generic points (or just generic points if the group is clear from the context). A generic 
stabiliser always exists for the reductive group actions on smooth affine varieties, see e.g. 
[27, §7]. Moreover, for the isotropy representation (Go : 0i), x G 0i is Go-generic if and 
only if X is Go-regular and semisimple if and only if 3g(x) fl 0i is a CSS in 0i. 

In the following theorem, we write 0^ in place of 3g(^) (^ G 0), and 0^ := 0^ fl Qij. 

Theorem 4.10. (i) The degenerated isotropy representation {K = Goo x A'oi : V = 0iooc0ii) 
always has a generic stabiliser. More precisely, let ^ G 0io be a Goo-generic point for the (little 
isotropy) representation (Goo ■ 0io) and let r] be a generic point in the G^^-module q\i. Then 
V = (^, 77) is a K-generic point in V and g.s.{K : V) = g.s.(GgQ : 0^ J x (0oi)''- 

(ii) trdegF(l^)^ = dim0io//Goo + dim0fi//G^o. 

(iii) F(r)^ zs the fraction field of ¥[V]^. 

Proof, (i) As explained above, ^ is semisimple and Cio := 0io is a CSS in 0io. Then 

(4-2) 011 = [0ol,e]©Sn• 

Note that 0QO © 011 is a Z2-grading of the reductive Lie algebra 0°^^ fl 0^. Let c be a CSS in 
0^1 and let ?7 G c be a Ggg-generic point. The stabiliser of 77) in Hs determined by the 
following conditions: 

= {(so,Si) G 000 X 0^1 I [so,e] = & [so,v] + K^] = 0}. 
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Hence sq g q^q. Then the second equation and (4-2) show that [so,r]] = and = 0; 

that is. So G (floo)'' ^^id si G ggi- Thus, 

Note that this stabiliser does not essentially depend on rj). If ^' G Cio and 77' G c are some 
other generic points, then 1'^^''') = t^^''^'\ In view of this observation, to prove that t^^'^'^ 
is a generic stabiliser, it is sufficient to show that the i^T-saturation of Cio oc c C 0io oc Qu is 
dense. 

By our constructions, there are the following relations for Cio and c: 

- Cqq-c is dense in gf^ ; 

- iVor(^', V) = {^', [Boi, + V) and [goi, = [floi, if ^' e Cio is generic; 

- Goo'Cio is dense in 010- 

The first two relations and the fact that Gqo does not affect Cio = flu show us that (Ggo ^ 
^oi)"(cio c) is dense in Cio oc gn. Then the last relation implies that Goo"(cio ocgn) is dense 
inV. 

(ii) By (i) and the Rosenlicht theorem [27, 2.3], we have 

dimg.s.(i^:\/) = dimg^i + dimg.s.(G^o ■ Sii) = 

dimg^i + dimg^g - dimgf^ + trdegF(gf J'^o". 

Since gf^ is an orthogonal Ggo'^'^dule, it follows from [12] that the field F(gf]^)'^oo is the 
fraction field of the algebra F[gf J'^w. Hence trdegF(gf^)'^oo = dirngf^/GgQ. Then using 
again the Rosenlicht theorem, we obtain 

trdegF(y)^^ Rosenlicht ^^^y _ dim + dimg.s.(i^ : V) = 

( dimgii + dimgio)-( dimgoi + dimgoo) + ( dimg^i + dimg^Q - dimg|i + dimgfi/G^o) = 

(dimgio - dim goo + dimg^g) + dimgfi/G^o = dimgio/Goo + dirngfi/G^Q, 

where the underlined terms in the second line are cancelled out, in view of Lemma 4.6 
applied to ^ G gio. 

(iii) The proof of [17, Lemma 2.6] applies in this situation. □ 

Corollary 4.11. The following conditions are equivalent: (i) trdegF(K)^ = dimgio/Goo/ (ii) 
011 = O/c' almost all ^ G gio, (iii) any CSS Ciq C gio is also a CSS in gio © gn, (iv) F[V]'^ ^ 
Flgio]^"". 

Corollary 4.12. It is always true that trdegF(V^)'^ = dimgu/Go* = trdegF(gu)'^o^ 

Proof. Choose a CSS Cio C gio. By [11], dimgio/Goo = dimcio and dirngf^^/Goo is the 
dimension of any CSS in g\^. Without loss of generality, we may assume that G Cio and 
3b(0 = Iisi^w)- Therefore, if t) is a Cartan subspace in gf^^ = 3g(cio)ii, then cio © f) is a CSS 
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in Qi^. That is, the right-hand side in Theorem 4.10(ii) is exactly dim /Go*- The second 
equality is a general property of orthogonal representations of reductive groups [12]. □ 

There is a general 'contraction procedure' for obtaining i^-invariants in ¥[V] or F[y*]. 
Recall that V and V* coincide as vector spaces and are isomorphic as Goo-iriodules. Only 
the Ai'oi-actions on them are different. Therefore, F[V]^ and F[V*]^ can be regarded as 
subalgebras of F[gio © gii]*^"". Let F[gio © Sii](a,b) denote the space of bi-homogeneous 
polynomials of degree a with respect to gio and degree b with respect to gn. Any ho- 
mogeneous polynomial / G F[gio © gn] of degree n can be decomposed into the sum of 
bi-homogeneous components / = YlT=kfi' where fi E F[gio © gii](n-j,j)- Assuming that 
fk,fm 7^ 0, we set /' := fk and /. := /„. That is, /* is the bi-homogeneous component 
having the maximal degree w.r.t. gio. 

Proposition 4.13. Suppose that f G F[gi^,]'^''* is homogeneous. Then 

/• G F[gio (X gii]^ and f, G F[gii ex gio]^. 

Proof. The proof is similar to that of [17, Prop. 3.1], which was designed for the adjoint 
and coadjotnt representations of K. For convenience, we repeat it here. Clearly, each 
bi-homogeneous component fi of / is Goo -invariant. For ^ ^ floi/ let V^j. denote the corre- 
sponding derivation of F[gi^]. (Here we regard the space Qu as Go^ -module, i.e., x is an 
element of the Lie algebra go^.) Since adx(gio) C gn and adx(gii) C gio, we have 

2)..(F[giJ(,,b)) cF[gi,] 

(a+l,b-l) {a-l,fe+l)- 

Of course, if a = or 6 = 0, then the summand with index a — 1 or 6 — 1 does not occur in 
the right-hand side. One can write = D+ + D~, where 

(a-l,fe+l)- 

Since = 0, we have D+(/') = and D-{f,) = 0. It also follows from (4-1) that D+ 

(resp. T)~) is the derivation corresponding to x, as an element of gg^^ C t, in the algebra 

F[goi oc gii] (resp. F[gii oc gio]). □ 

Corollary 4.14. The algebras ¥[V]^ and ¥[V*]^ are bi-graded. 

Proof. The previous proof shows that if / G F[V]^^, then also /' G F[l^]^. Then one 
considers / — /', etc. □ 

Remark 4:.15. Set gr •(F[gu]^"*) = {/• | / g F[gu]^"*} c ¥[V]^. We already know that 
trdegF(l^)^ = trdegF(gu)^"* = dim(gu//GoO and ¥{V)^ is the fraction field of ¥[V]^\ 
Therefore, one might expect that gr •(F[gi^,]'^"*) = F[y]^ in good cases. Indeed, it often 
happens that there is a set of basic invariants /i, . . . , /m G IF[gi^]'^"* such that /',..., 
are algebraically independent. But, unlike the case of the coadjoint representation of K, 
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this does not guarantee that /*, • • • , generate F[y]^^ (cf. [17, Theorem 4.2]). True gen- 
erators of F[y]^ can have smaller degrees. The reason is that, for the A'-module V, the 
complement of the set of K-regular points may contain a divisor. (By [17, Theorem 3.3], 
this cannot happen for the coadjoint representation of a Z2-contraction.) 

5. Invariants of certain degenerated isotropy representations 

We continue to work with a quaternionic decomposition (2-2). In Section 4, we considered 
a good situation in which a CSS Cio C 0io is also a CSS in g^. This is equivalent to that 
3g(cio) n = CiQ. Using Theorem 4.5, one then obtains a description of F[gio ocflu]^"^, 
etc. In this section, we consider a less restrictive condition that Cio contains Go*-regular 
elements of gi^. In other words, 

(5-1) 30(cio) n Qu is a CSS in g^. 

Then D := 3g(cio) fi Qu = Cio © a for some subspace a C gn. Recall that the generalised 
Weyl group for the CSS C g^ is = No^{d)/Zo^{d), where No^{d) = {s e Go* I s-d C d} 
and Zo^id) = {s E Go* | s-x = x Vx G d}. Set m = dim 0. 

Obviously, the involution a2 preserves d and the corresponding eigenspaces are Cio and a. 

Lemma 5.1. (72 = o-2|j, normalises the group W C GL{d), i.e., d-2Wd-2^ = W. 

Proof, li g E Go*, then 0-2(5') ^ Co* as well, and 172(5') acts on g as composition 0-2050 0-2^- 
Therefore, if 5 G Nq^{D), then also 0-2(5) ^ ^o*(^')- D 

Set X = Go*-Cio C Qu. Since Gqo-Cio = Qio, we may as well define X as Go*-gio. 

Theorem 5.2. If condition (5-1) is satisfied, then X is an irreducible complete intersection in gi* 
and the ideal of X in F[gu], J(X), is generated by some basic invariants in F[gi*]'^°*. 

Proof. Clearly, X is irreducible. Since Cio contains Go*-regular elements of gi* and Go^,■^ = 
Qu, we have dim gij, — dim X = dim d — dim Cio = dim a. 

We know that 0-2 G GL(J)) is of finite order, normalises W, and has regular eigenvectors 
(in Cio). By [23, 6.3-6.5], this implies the following: 

- The centraliser Wiq of c72 in is reflection group in the (72-eigenspace Ciq. 

- F[cio]^^° is generated by the restrictions to Cio of some basic invariants in F[d]'^. 
Namely, a set of basic invariants {/i, • • • , /m} can be chosen such that each fi is a 0-2- 
eigenvector, say 0-2 (/i) = Eifi, where Ei G {1,-1}. Then F[cio]^^° is freely generated by 
fi\cw such that El = 1. In particular, the restriction homomorphism ¥[d]^ — )■ F[cio]'^^" is 
onto. 

- The eigenvalues of 0-2 in are ei, . . . , Em- 
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It follows that there is a set of basic invariants {fi, . . . , fm} in F[^]'^ such that fi, . . . , fk 
vanish on cio, whereas the restrictions of fk+i, . . . , /m to Cio freely generate F[cio]'^^°. More- 
over, here m — k = dim Cio and k = dim a. 

Since Ffgi^]*^"* ~ F[0]^, there are also k basic invariants in F[gi^]'^°* that vanish on 
X. Let Fi denote the Go* -invariant corresponding to fi. Then Fi, . . . , is a regular 
sequence in ¥[gi^] and V(Fi, . . . , F^) is an unmixed variety of codimension dim a. Fur- 
thermore, X C V(Fi, . . . , Fk) and codimX = k. Hence X is an irreducible component of 
V{Fi, . . . , Fk). Our goal is to prove that the ideal (Fi, . . . , Fk) is prime. The first step is 
prove that V{Fi, . . . , Fk) is irreducible. 

Recall that the morphism tt : gi^ — )■ QuZ/Gq^ is equidimensional, and a generic fibre of vr 
is irreducible, since it is a (closed) Go*-orbit. If Z is a Go*-stable closed subset of gu, then 
7i{Z) is also closed [27, 4.4]. The previous discussion on invariants of W and Wio shows 
that 7r(V(Fi, . . . , Fk)) = it{X) = 7r(cio) is an affine space of dimension m — k. 

Let Y be an irreducible component of V(Fi, . . . , F^). Since n is equidimensional, 
dimX = dimF, and 7r(F) C 7r(V(Fi, . . . , F^)) = 7r(X), we must have 7r(X) = 7r{Y). 
Moreover, for generic rj = n{x) G Tr{Y) {x G Cio), we have Tr^^{r]) = Gq^^-x C X. Thus, 
Y = X and X = . . . , Ffc) is irreducible. 

By [10, Lemma 4], the primeness of the ideal (Fi, . . . , Fk) will follow from the fact that 
there is a point y e X = V{Fi, . . . , Fk) such that the differentials dFi, . . . , dFk are linearly 
independent at y. Write {dF)y for the value of dF at y. By assumption, X contains a Gq*- 
regular semisimple element x, and it is known that even (dFi)^:, . . . , {dFm)x are linearly 
independent [11]. □ 

Remark 5.3. Using the fact that (dFi)^., . . . , {dFk)x are linearly independent for any Gq^- 
regular point of x E X, one easily proves that X is non-singular in codimension 2 and 
therefore X is normal. (A similar argument in the context of adjoint representations is 
found in [19, § 5].) 

Keep the previous notation, i.e., m = rk (G/Gq^), X = Go*-Cio, and Fi, . . . , F^ G F[gi*]'^°* 
are the basic invariants such that J(X) = (Fi, . . . , F^) and ¥[Xf'>* = W[Fk+i, . . . , F^]. 
Recall that m — k = dim Cio and k = dim a. We wish to describe the invariants of the 
isotropy representation {K : V = Qiqoc Qu), which is a contraction of (Go* : gi*). 

As in Section 4, we consider bi-homogeneous components of Fj with respect to the sum 
01* = 010 ffi 011- Recall that F' stands for the bi-homogeneous component of maximal 
degree with respect to gio, and F' G F[gio oc g^^jGooKiVoi p^-^p ^ i^,. 

Theorem 5.4. Suppose that condition (5-1) is satisfied. 

(i) The field F(gio oc gn)^"^ is generated by the coordinate functions on gio and the polynomials 
F',i = l,...,k. 
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(ii) Suppose that (dFi)e, . . . , {dFk)e i^e linearly independent for a Goo-regular nilpotent element 
e G 010- Then 

- the algebra F[gio oc gn]^"! is freely generated by the coordinate functions on Qio and the 
polynomials F', i = 1, . . . ,k; 

- the algebra F[gio oc g-^-^jGooxA^oi is freely generated by the basic invariants in F[gio]*^°° and the 
polynomials F', i = 1, . . . ,k. 

Proof. We are only interested in basic invariants Fi with i ^ k. Let degFj = n^. Since Fj 
vanishes on X and gio = Goo'Cio C X, the bi-homogeneous component of degree (nj, 0) is 
trivial. 

By (5-1), contains Go^-regular semisimple elements. If s G gio is such an element, 
then (dFi)s, . . . , {dFk)s are linearly independent. The fact that (dFj)^ is nonzero for some 
s e implies that each Fj has a bi-homogeneous component of degree (rij — 1, 1), which 
thereby is equal to F'. Since this bi-homogeneous component is linear with respect to gn, 
it can be written as 

(5-2) F'{yo,yi) = K{Ti{yo),yi), 

where : goi — )■ gn is a polynomial mapping of degree rij — 1. Then 

idFi)y, = Tiiyo). 

As F* is Goo-invariant, the mapping J^j must be Goo-equivariant. It follows from Equa- 
tions (5-2) and (4-1) that the A^oi-invariance of is equivalent to that J^i{yo) commutes 
with yo- Recall that gn is a A'-stable subspace oiV = gio oc gn and the induced action of 
A^oi ori V/qu ^ gio is trivial. Consider the polynomial mapping 

^ : 010 oc gn ^ (^/Sii) x F'^ ^ gio x F^ 

defined by 

i^{yo,yi) = (2/0,^1 (2/o,yi), ••• ,^'(2/0,2/1)) = {yo,f^iJ^iiyo),yi),---,f^{J^k{yo),yi))- 

Clearly the A^oi -action on (V/gu) x F'^ is trivial and tjj is A^'oi-equivariant. Our ultimate 
goal is to prove that tjj is the quotient by Nqi. But we can perform it under the additional 
hypothesis in part (ii). Without extra hypotheses, we can only get the result on the field 
of Ai'oi -invariants. 

(i) Let ils C gio be the open subset of semisimple elements that are both Go^-regular 
and Goo-regular. Condition (5-1) means that 7^ 0. Since the vectors J'liQi ■ ■ ■ )-^fc(C) 
are linearly independent for all ( G fi^/ we have x F''' c Im tjj. Hence is dominant. 
For {yo, 2:1, ... , Zk) G gio x F'', the fibre 

^'"^(yo, zi,..., Zk) = {{yo, yi) I K{Fj{yo), yi) = Zj, 1 ^k} 
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is an Ai'oi-stable affine subspace of Qiq oc Qu. Furthermore, if yo e Qg, then 

dimilj~\yo, zi,...,Zk) = dimgn - k. 

On the other hand, if {yo, yi) G ip^^ijjQ, ^i, • • • , z^), then 

^or(z/o,yi) = (z/o,Z/i + [0oi,yo]) C^"^(i/o,2:i,---,^fc) 

and dim[goi,?/o] = dim[gii, yo] = dim flu - dim3g(yo)ii- If Z/o e f^^, then 3g(?/o)io is a CSS in 
010 and 3g(?/o)i* is a CSS in g^. Hence dim3g(?/o)ii = dim a = k and 

dimA^or(2/o,2/i) = dimgn - k = dim?/^"^(?/o, ^^i, • • -.Zk). 

As the orbits of a unipotent group on affine varieties are closed [21, Theorem 2] and iso- 
morphic to affine spaces, we obtain 'ip~^{yo, -Zi, . . . , ^a:) = Noi-{yo, yi) for yo ^ ^s/ i-e., almost 
all fibres of il) are just Ai'oi -orbits. 

By [27, Lemma 2.1], this means that the coordinates on gio and F*, ■ ■ ■ ,F' generate the 
field F(0ioocgii)^»\ 

(ii) We wish is to apply the Igusa lemma [16, Lemma 6.1] to ifj. As the target variety 
010 X F'^ is normal and generic fibres of are iVoi-orbits, it remains to check that Im^ 
contains an open subset of 0io x F''' whose complement is of codimension at least 2. 

Let c 010 be the open set of all Goo-rcgular elements (i.e., not only semisimple ones!). 
It follows from [11] that codim (010 \ f2) ^2. Hence it would be sufficient to have that 
f2 X F^ C Im ^. This, in turn, would follow from the fact that 

(O) the differentials (dFi)^, . . . , {dF^jr, are linearly independent for all rj &Vt. 

An obstacle is that if 77 G fi, then 77 is not necessarily Go*-regular. Therefore, we need the 
assumption that (dFi)e, . . . , {dFk)e are linearly independent for a nilpotent e E d. Then a 
standard deformation argument guarantees us condition {<>). 

Thus, the assumptions of the Igusa lemma are satisfied and the assertion on the algebra 
of A^'oi -invariants follows. The assertion on Goo 1^ A'^gi -invariants stems form the fact that 
the J^'s ^re already Goo -invariant. D 

Remark 5.5. 1) It might be true that (5-1) already implies that (dFi)e, . . . , {dFk)e are lin- 
early independent for a Goo-regular nilpotent e G 0oi. Such a phenomenon is observed in 
several examples. But we unable to either prove or disprove this as yet. 

2) Although the algebra F[gio ocgn]^*" is polynomial in Theorem 5.4(ii), the quotient 
morphism 

T^Noi ■ 010 OC 011 (0ioocgii)/iVoi ~ gio x F^ 

is not equidimensional (unless k = 0, i.e., we are in the situation of Theorem 4.5). Yet it 
might be true that the quotient morphism hy K = Goo x A^oi 

ttk ■■ 010 oc 011 (0ioocgii)//(Goo x iVoi) ~ (gio//Goo) x F*^ ~ F"" 
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is equidimensional. However, our methods for verifying equidimensionality require an 
information on the dimension of the linear span of {(dFi)^,, . . . , {dFk)v} for nilpotent ele- 
ments of f G 010, which is difficult to infer in general (cf. proof of [17, Theorem 5.3]). 

There is a sufficient condition that guarantees that Theorem 5.4 applies in full strength. 

Proposition 5.6. Let e G Qio be a GoQ-regular nilpotent element. If e is also Go^-regular as 
an element of Qi^,, then then all the assumptions of Theorem 5.4 are satisfied, and hence both 
F[gio oc flii]^"^ and F[gio oc g^^J'^ooKAToi polynomial algebras. 

Proof. Let {e, h, /} C be a normal s[2-triple. If e is both Goo-regular and Go*-regular, 
then the same is true for the semisimple element e + / G gio [H]- Hence condition (5-1) 
is satisfied. It was also explained above that the Go*-regularity of e implies the linear 
independence of the differentials for all basic invariants in F[gi^]'^°*. □ 

We consider below some applications of Theorem 5.4(ii). 

Example 5.7. Let {i!), /i} be a canonical triple of involutions of g, i.e., ^9, ^9' are of maximal 
rank and /i is quasi-maximal. The corresponding quaternionic decomposition 

000 ; 001 

g = ■^■■(^■^ i9 

010 i 011 

has a number of good properties, see Proposition 2.2 and Remark 2.3. We wish to describe 
the algebras of invariants for all degenerated isotropy representations. By Corollary 3.5, 
there are at least four coincidence of CSS; hence the invariants for the respective degen- 
erated isotropy representations are given by Theorem 4.9. If 13 is inner, i.e., {-(9, 'd', ^} is 
actually a triad, then all little and big CSS are Cartan subalgebras and the invariants of all 
six degenerations are described by Theorem 4.9(iii). But, if i!) is not inner, then there are 
two degenerated isotropy representations, (GooX^oi : 0io ocgn) and (GooX^io : 0oi ocqu), 
where Theorem 4.9 does not apply. We show that the hypotheses of Theorem 5.4 are sat- 
isfied for them, and hence the their algebras of invariants are also polynomial. 

Assume below that '& is not inner, so that and are not conjugate. Then ko ^ and 

dim goo = — dimgoi = dimgio = dim goo + h, dimgn = dim goo + rkg 

(see the proof of Prop. 2.2). We also have rkg'' = rkg''' = ki (Proposition 1.3). As ^9 is 
of maximal rank, any CSS Cu C Qu is a Cartan subalgebra of g. Recall that Ffgi^]"^"* ~ 
F[cu]^, where W is the generalised Weyl group. In this case, W coincides with the usual 
Weyl group of g with respect to Ci^. Hence F[gi^]'^o* ^ F[g]'^. Let Cio be a CSS in gio. Then 
dimcio = /ci = rkg — kQ. Hence Cio cannot be a CSS in gi^. However, Cio does contain 
regular semisimple elements of g (see the proof of Prop. 2.2), i.e., (5-1) is satisfied. 
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In this situation, the number k occurring in Theorems 5.2 and 5.4 is ko and basic invari- 
ants Fi, . . . , Ffcg are precisely the basic invariants in Ffg]*^ of odd degrees. (Recall that we 
have defined ko as the number of even exponents of q.) If e G gio is Goo-regular nilpotent, 
then e is also regular nilpotent in g^o = Q^' > since this involution of gy,o is of maximal rank, 
see [1]. Therefore, the hypothesis of Theorem 5.4(ii) can be restated as follows: 

Let ?9 be an (outer) involution of maximal rank and e a regular nilpotent element of g'^. Let 
Fi, . . . , Fko G Ffg]*^ be the basic invariants of odd degrees. Then (dFi)e, . . . , {dFko)e i^e linearly 
independent. 

We verify this for all relevant simple Lie algebras. 

ai) g = 5(2n+i and = S02n+i- Here e is regular in the whole of g. 

^2) g = sbn and = S02n- The partition of e is (2n — 1,1), i.e., e is a subregular nilpotent 
element of g. Let fi be a basic invariant of degree i {i = 2, 3, ... , 2n). It is known that, 
for a subregular nilpotent element e, {df2n)e = (i.e., for the basic invariant of maximal 
degree), while the remaining differentials are linearly independent [20, Lemma 5.1]. This 
remaining set contains all basic invariants of odd degree. 

as) g = S04„+2 and = S02n+i x S02n+i- The partition of e is {2n + 1, 2n + 1) and ko = 1. 
The only basic invariant of odd degree is the pfaffian P. One verifies that, for a nilpotent 
element v, (dP)^ = if and only if the partition of v has at least three nonzero parts [20, 
Lemma 4.1.1]. 

ai) g = Eg and = spg. Here /cq = 2, degFi = 5 and degF2 = 9. As e appears to be 
subregular in g, we can again use [20, Lemma 5.1]. 

Thus, in case of canonical decompositions, we can describe the algebras of invariants 
for all six degenerated isotropy representations. 

Example 5.8. Let g be of type Eg. According to [9, Table 1], there are three commuting 
involutions with the following fixed-point subalgebras: 

ai,(j2: F4; 0-3: D5 ©ti. 
As {cTi, (T2} is supposed to be a dyad, this can also be verified using the restricted root 
system for E6/F4, which is reduced and of type A2 (see Section 2). Here goo = sog and the 
goo-iriodules gij are: 

goi - 010 - R^4' 011 - R^i + Ro/ 

where w/s are fundamental weights of sog and Ra stands for the simple module with 
highest weight A. Hence dim goi = dimgio = 16 and dim gn = 10. Here dim Cio = dimcoi = 
1 and dim Cn = 2. By Theorem 3.7, Cn is also a CSS in gi^ or g^i; and these are the only 
coincidences of CSS here. In view of these coincidence and Theorem 4.9(ii), the algebra of 
invariants for two isomorphic degenerated isotropy representations, (Goo ^ ^01 : gii oc gio) 
and (Goo x ^10 : gii oc goi), is readily described: 

F[(R^, + Ro) oc R^jSpin,Kexp(RS,J ^ ^[R^^ + R^jSpin^^ 
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Therefore, this algebra has the basic invariants of degree 1 and 2. The representation 
(5-3) (Sping K exp(R:,J : (R^,+Ro) oc R^J 

is a degeneration of the isotropy representation of the symmetric space E6/F4, i.e., of 
(F4 : R), where dim R = 26 and the degrees of basic invariants are 2 and 3. The other 
degeneration of the same isotropy representation is 

(5-4) (Sping K exp(R^J : R^, cx (R^, + Ro)). 

Here we can exploit the fact that any Goo-regular nilpotent element in 010 ~ R^i appears 
to be Go* -regular in gi^, ~ R. Indeed, using the Satake diagram for the symmetric pair 
(F4,509), one verifies that the Sping-regular nilpotent element of gio belong to the nilpo- 
tent F4-orbit O, denoted A2. On the other hand, using the Satake diagram for the sym- 
metric pair (Ee, F4), one verifies that the F4-regular nilpotent element of gu belongs to the 
nilpotent Eg-orbit O, denoted 2A2. It is not hard to verify using the respective weighted 
Dynkin diagrams that (9 fl f 4 = C 

Since dim Cu — dim Cio = 1, X = Gq^-Qio is a hypersurface in gi*, and this hypersurface 
must be the zero set of the basic invariant of degree 3 in F[R]^''. Finally, applying Theo- 
rem 5.4 and Prop. 5.6 shows that F[R^, oc (R^,+Ro)]^P'"9''''^p('^-4) is a polynomial algebra 
with basic invariants of degree 2 and 3. 

Thus, the dual representations (5-3) and (5-4) have polynomial algebras of invariants, 
but the degrees of basic invariants are different. 

Set V = (R„i+Ro) ocR^4, V* = R„^ oc (R^.+Rq), and K = Sping k exp(R^J. Since 
dim V// K = dim V* // K = 2 and K has no rational characters, one easily derives that both 
morphisms ttv : V ^l/K ^ and vry* : V* ^ V* // K c:^ A? are equidimensional; hence 
F[V] is a free F[V']^'' -module, and likewise for V*. 

6. Problems and observations 

6.1. In [17], I suggested that the coadjotnt representation of a Z2-contraction of q al- 
ways has a polynomial algebra of invariants. However, recent work of O. Yakimova [28] 
demonstrates that this is not always the case. Therefore, there ought to be quatemionic 
decompositions of a simple Lie algebra g such that some degenerated isotropy represen- 
tations (Goo X : 0a oc g^) do not have a polynomial algebra of invariants. On the other 
hand, results of Sections 4 and 5 show that in many cases the algebra F[gQ, oc g^]'^""'^^'' is 
polynomial. This raises the following 

Question 1. What are more precise necessary and /or sufficient conditions for 

^Qa OC g^]^o"^^« to be polynomial? 
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In Section 4, we proved that if a CSS Ca C Qa is also a CSS in © g^, then ¥[ga oc g^] is a 
free Ffg^ oc g^]*^'"'^^'^ -module, i.e., the quotient morphism 

vr : go oc g^ ^ (ga oc gj)//Goo k iV^ ~ A™ 

is equidimensional. This raises our next 

Question 2. What are more precise necessary and/ or sufficient conditions for ¥[ga oc g^] 
to be a free Wlg^ oc g^]'^""''^'' -module? 

Of course, both Questions are prompted by the fact that the respective properties are 
always satisfied for the initial (big) isotropy representations, see Section 1. 

6.2. Let Q be an algebraic group with L\e{Q) = q. Suppose that a is an involution of 
Q and Qo is the identity component of Q"^. Let H be any subgroup between Qo and Q'^ 
such that Q/H is connected (this is always the case if Q is connected). Then the alge- 
bra, V{Q/H), of left invariant differential operators on the homogeneous space Q/H is 
commutative [22], [4]. Let q = qo © qi be the corresponding Z2-grading. The following is 
proved in [5, Lemma 4.2]: 

If the algebra §(qi)^ = F[q^]^ is polynomial, then so is V{Q/H). 

In fact, there is a canonical linear bijection between the two algebras that transforms the 
free generators of the former to the free generators of the latter. 

Therefore, our results on quaternionic decompositions provide a description of invari- 
ant differential operators for certain degenerations of symmetric spaces of G. Recall that 
the group Kqi = Gqo x ^oi can be regarded as the identity component of a symmetric 
subgroup in two different ways: 

a) Kqi is the identity component of G(a2Y^, and the corresponding isotropy represen- 
tation is (Koi : Sioocgii); 

b) Kqi is the identity component of G{a-i)°'^, and the corresponding isotropy represen- 
tation is {Koi : giiocgio). 

Taking into account that (gio oc gn)* = gn oc gio, we obtain the following: 

Claim 6.1. If F[gio oc gn]^'" is a polynomial algebra, then so is V{G{a-i) / Kqi); 
if F[gii oc gio]^"^ is a polynomial algebra, then so is V(G{a2) / Kqi). 

It might be interesting to explore a relationship between the joint eigenfunctions for 

V{G/Gq^) and P(G(a3)/i^oi)- Also, if Cio is a CSS in goi © gn, then F[gio ocgn]^'^'"! ~ 
IF [gio] '^""z which seems to suggest that there might be a direct relation between the poly- 
nomial algebras of differential operators V{G{a3)/KQi) and ©(G^q/Goo)- 
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6.3. In [5], Gonzalez and Helgason determined the algebra of invariant differential op- 
erators for an interesting non-reductive symmetric pair {Q,Q'^)- In their exposition, the 
(non-connected) group Q is the semi-direct product of the orthogonal group 0„ = 0{V) 
and the standard 0(V')-module V , i.e., using our notation, q = so{V) k V and Q = 0{V) k 
exp iV"-). Then they consider an involution aoiQ such that = ( ( ) x exp ( V" ) ) x ( ^2 ) / 
where V = l^i © V2 is an orthogonal direct sum. Set p = dim Vi, so that n — p = dim V2. 

Our observation is that Q/Q'^ is a degeneration of the symmetric space 0„+i/Op+i x 
On-p, and this degeneration is related to a quatemionic decomposition of g = sOn+i- To 
get this, we assume that so„+i = 5o{V) consists of usual skew-symmetric matrices of 
order n + 1 with respect to a certain basis of {n + 1) -dimensional space V and consider the 
involutions cTj {i = 1, 2, 3) defined by the diagonal matrices si, S2, S3 E 0{V) : 

si = diag(l, . ^. , 1, -1, . , -1 ), S2 = diag(l, . , 1, -1, . ^. , -1 ), S3 = S1S2. 

p+l n—p p n— p+1 

Let Vi (resp. V2) be the linear span of the first p (resp. last n—p) basis vectors. Then 
goo = so{Vi) X 30(^2) and the whole quatemionic decomposition is: 

so{Vi)xso{V2) \ Vi 

0-2 

(i.e., ^ V2 as 0oo-module, etc.) Here V = Vi ® V2 ® W = V ® ¥, 0^1 = so(Vi © 
F) X 30(^2), 0^"^ = so{V2 © F) X 5o(Vi), and 0^^:' = so{V). In our notation, the Lie algebra 
L\e{Q) = q considered in [5] is nothing but 5(0-3) ; and Theorem 3.1 in [5] essentially means 
that, for H := (0(Vi) x 0(^2)) x exp(Vi"), the algebra ¥[¥2 oc (V^i ® 1/2)]^ is polynomial, of 
Krull dimension mm{p + l,n — p}. In our setting, the dimension formula is explained by 
Corollary 4. 12 and the fact that rk (SO„+i/SOp+i x SO„_p) = inm{p + l,n — p}. The identity 
component of H is 

i^oi = (SO(V^i) X §0(^2)) X exp{V,^), 

and one can prove that, for this degenerated isotropy representation, we have a material- 
isation of the phenomenon of Remark 4.15, i.e., gr •(F[0u]'^o*) = ¥[¥2 oc {Vi (g) V2)]^°\ Note 
that Go* = §>0{Vi © F) X §0{V2) and the basic invariants of F[0u]'^«* have degrees 

2, 4, . . . , 2 min{p + l,n — p} ifp+l^n— p^ 
2, 4, . . . , 2p, p + 1 if p + 1 = n — p . 

Therefore, the same is true for ¥[¥2 oc (Vi © ^2)]^°^ Since dim Cio = 1 and dim Ci^, = min{j9 + 
l,n — p}, the coincidence of CSS occurs only iin — p = 1. 
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The advantage of using quaternionic decompositions is that we immediately get other 
interesting possibilities. For the dual i^oi-n^odule {Vi V2) (XV2, we may compare the 
Cartan subspaces Cn cVi^V2 and Ci*. 

• Since dim Cn = mm{p, n — p} and dim Ci* = min{p + 1, n — p}, we have a coincidence 
of CSS if and only iin-p^p, and then ¥[{Vi » V2) oc ¥2]^°' ~ ¥[Vi ® V2f'^^^''>''^'^^^^l The 
latter is a polynomial algebra whose degrees of basic invariants are 

2,4,..., 2(n — p), iin ~ p < p; 2, 4, . . . , 2p — 2, p, if n — p = p. 

• For p+1 = n — p, one can notice that a Goo-regular nilpotent element in Qu ~ Vi ® 

is actually regular in the whole of g. Hence, Proposition 5.6 applies to Kqi : (Vi ® V2) oc V2 
and hence ¥[{Vi ® V2) oc V2]^°^ is polynomial. 

• Even for p+1 < n — p, we can prove using the 'contraction procedure' for Goi,- 
invariants that F[(Vi ® V2) oc V2]^^"^ is a polynomial algebra. The details will appear else- 
where. 

As explained in Subsection 6.2, this implies that V{G{a2) /Kqi) is a polynomial algebra, 
where G = §0{Vi © V^2 © F) and G{a2) = {SO{Vi) x §0(^2 © F)) k {Vi O {V2 © F)). 
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